Multi-Mask Label Mapping for Prompt-Based Learning

Prompt-based Learning has shown significant success in few-shot classification. The mainstream approach is to concatenate a template for the input text transform classification task into cloze-type where label mapping plays an important role finding ground-truth labels. While current methods only use contexts one single input, it could be crucial if wrong information contained text. Specifically, proved recent work that even large language models like BERT/RoBERTa make decisions heavily dependent on specific keyword regardless of or context. Such word referred as lexical cue and misleading included instance will lead model prediction. We propose multi-mask prompt-based with Multi-Mask Label Mapping (MMLM) reduce impact cues by allowing exploit multiple cues. To satisfy conditions learning, augmentation proposed are gradually excluded through training. demonstrate effectiveness MMLM both theoretical analysis empirical studies, show outperforms other existing approaches.